A knowledge-based task planning approach for robot multi-task manipulation

Abstract Task planning is a crucial component in facilitating robot multi-task manipulations. Language-based task methods offer practicality receiving commands from humans real-life scenarios and require only low-cost labeled data. However, existing often rely on sequence models for planning, which primarily focus mapping language to sequences of sub-tasks while neglecting the knowledge about tasks objects. To overcome these limitations, we propose knowledge-based approach called Recurrent Graph Convolutional Network (RGCN). It devised with novel structure that combined GCN (Kipf Welling International Conference Learning Representations (ICLR), 2017) LSTM (Hochreiter chmidhuber Neural Comput 9 (8): 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735 ) enables it leverage graph data historical predictions. The experimental results demonstrate our achieves impressive success rate $${95.7\%}$$ 95.7 % , surpassing best baseline method significantly, $${78.7\%}$$ 78.7 . Furthermore, evaluate performance manipulation across specific set 20 within simulated environment. Notably, RGCN pre-trained primitive exhibits highest compared state-of-art learning methods. Our proven be significant language-conditioned qualified instructing robots manipulation.